A Two-stream Fusion Framework for Lidar and Hyperspectral Imagery

Data fusion can significantly increase accuracy of automated classification in remote sensing applications by combining data from different types of sensors. Particularly for hyperspectral imagery (HSI), complementing the hyperspectral data with topographical information in the form of a Digital Surface Model (DSM) generated by LiDAR data is promising to address problems with artifacts or distortion in difficult areas. In this paper we introduce a novel framework for combining the HSI and LiDAR data, which enables handling identified objects as uniform entities rather than as independent pixels. Further contributions include an initial spectral unmixing step that segregates noise and significantly improves the benefit of adding LiDAR, as well as the application of ensemble learning in the form of Random Forest algorithms that inherently support feature selection.